Method and apparatus providing network coding based flow control

ABSTRACT

A method, apparatus and computer program product for providing network based flow control is presented. A linear combination of packets to transmit from a transmit queue is determined. The linear combination of packets is transmitted across a network using a sliding window protocol. An acknowledgement (ACK) is generated, wherein a packet is acknowledged when a receiving node receives the linear combination of packets and determines which packet of the linear combination of packets has been newly seen.

GOVERNMENT RIGHTS

This work is supported by subcontract number 18870740-37362-C issued byStanford University and supported by DARPA, NSF Grant No. CNS-0627021and subcontract number 060786 issued by BAE Systems and supported byDARPA and SPAWARSYSCEN under Contract number N66001-06-C-2020, andS0176938 issued by UC Santa Cruz supported by the United States Armyunder Award No. W911NF-05-1-0246, DARPA Grant No. HR0011-08-1-0008 andsubcontract number 069145 issued by BAE Systems and supported by theDARPA and SPAWARSYSCEN under Contract No. N66001-08-C-2013. Thegovernment has certain rights in the invention.

This invention was made with government support under Grant No.W911NF-07-1-0029 awarded by the Army Research Office, Grant NumberN66001-06-C-2020 awarded by the Defense Advanced Research ProjectsAgency, and Grant Numbers CNS0627021, CNS0721491, and CCF0634923 awardedby the National Science Foundation. The government has certain rights inthis invention.

BACKGROUND

Network coding has emerged as an important potential approach to theoperation of communication networks, especially wireless networks. Themajor benefit of network coding stems from its ability to mix data,across time and across flows. This makes data transmission over lossywireless networks robust and effective.

Linear network coding was originally introduced for the case oferror-free networks with specified link capacities, and was extended tothe case of erasure networks. The linear network coding solution doesnot require decoding at intermediate nodes and can be applied in anynetwork. Each node transmits a linear combination of all coded packetsit has received so far. This solution ensures that with highprobability, the transmitted packet will have what is called theinnovation guarantee property, i.e., it will be innovative to everyreceiver that receives it successfully, except if the receiver alreadyknows as much as the sender. An innovative packet is a linearcombination of packets which is linearly independent of previouslyreceived linear combinations, and thus conveys new information. Thus,every successful reception will bring a unit of new information. Thisscheme is shown to achieve capacity for the case of a multicast session.

The Transmission Control Protocol (TCP) was originally developed forwired networks. Since wired networks have very little packet loss on thelinks and the predominant source of loss is buffer overflow due tocongestion, TCP's approach of inferring congestion from losses workswell. In contrast, wireless networks are characterized by packet loss onthe link and intermittent connectivity due to fading.

SUMMARY

Conventional mechanisms such as those explained above suffer from avariety of deficiencies. In considering the potential benefits of theTCP-compatible network coding solution, the area of wireless links is ofparticular interest. It is well known that TCP is not well suited forlossy links, which are generally more prevalent in wireless systems. TCPperforms poorly on lossy links primarily because it is designed tointerpret each loss as a congestion signal. Adapting TCP for wirelessscenarios is a very well-studied problem. The general approach has beento mask losses from TCP using link layer retransmission. However, it hasbeen noted that the interaction between link layer retransmission andTCP's retransmission can be complicated and that performance may sufferdue to independent retransmission protocols at different layers. Moreimportantly, the benefits of approaches such as multipath opportunisticrouting which exploit the broadcast nature of the wireless medium, linklayer retransmission may not be the best approach. TCP wrongly assumesthe cause of link losses to be congestion, and reduces its transmissionrate unnecessarily, leading to low throughput. These problems of TCP inwireless networks are very well studied, and several solutions have beenproposed.

Embodiments of the invention significantly overcome such deficienciesand provide mechanisms and techniques that provide a new approach tocongestion control on lossy links based on the idea of random linearnetwork coding. Also introduced is a new acknowledgment mechanism thatplays a key role in incorporating coding into the congestion controlalgorithm. From an implementation perspective, a new network codinglayer between the transport and network layers on both the source andreceiver sides. Thus, requisite changes can be easily deployed in anexisting system.

A technique is described that incorporates network coding into TCP withonly minor changes to the protocol stack, thereby allowing incrementaldeployment. In the present invention, the source transmits random linearcombinations of packets currently in the congestion window. At the heartof the techniques is a new interpretation of ACKs wherein the sinkacknowledges every degree of freedom (i.e., a linear combination thatreveals one unit of new information) even if it does not reveal anoriginal packet immediately. Such ACKs enable a TCP-compatiblesliding-window approach to network coding. By way of the currenttechnique packet losses are essentially masked from the congestioncontrol algorithm. The algorithm therefore reacts to packet drops in asmooth manner, resulting in a novel and effective approach forcongestion control over networks involving links such as wireless links.The technique also allows intermediate nodes to perform re-encoding ofthe data packets. Simulations show that the algorithm, with or withoutre-encoding inside the network, achieves much higher throughput comparedto TCP over lossy wireless links. Also described is the soundness andfairness of the algorithm, as well as a queuing analysis for the case ofintermediate node re-encoding.

In a particular embodiment of a method in which a transmitting nodeperforms operations for providing network coding based flow control, themethod begins by determining a linear combination of packets to transmitfrom a transmit queue. The method also includes transmitting the linearcombination of packets across a network using a sliding window protocol.The method further includes receiving an acknowledgement (ACK), whereina packet is acknowledged when a receiving node receives the linearcombination of packets and determines which packet of the linearcombination of packets has been newly seen.

In a particular embodiment of a method in which a receiving nodeperforms operations for providing network coding based flow control, themethod begins by receiving a linear combination of packets across anetwork using a sliding window protocol. The method further includesdetermining whether a packet is newly seen. The concept of a newly seenpacket is defined later. Additionally, the method includes transmittingan acknowledgement (ACK), wherein a packet is acknowledged when thereceiving node receives the linear combination of packets and determineswhich packet of the linear combination of packets has been newly seen.

In a particular embodiment of a method in which an intermediate nodeperforms operations for providing network coding based flow control, themethod begins by receiving a linear combination of packets. The methodalso includes determining a linear combination of packets to transmitfrom a transmit queue. The method further includes transmitting thelinear combination of packets across a network using a sliding windowprotocol.

Other embodiments include a computer readable medium having computerreadable code thereon for providing a network coding based flow controlat a transmitting node. The computer readable medium includesinstructions for determining a linear combination of packets to transmitfrom a transmit queue. The computer readable medium also includesinstructions for transmitting the linear combination of packets across anetwork using a sliding window protocol. The computer readable mediumfurther includes instructions for receiving an acknowledgement (ACK),wherein a packet is acknowledged when a receiving node receives thelinear combination of packets and determines which packet of the linearcombination of packets has been newly seen.

Still other embodiments include a computer readable medium havingcomputer readable code thereon for providing a network coding based flowcontrol at a receiving node. The computer readable medium includesinstructions for receiving a linear combination of packets across anetwork using a sliding window protocol. The computer readable mediumfurther includes instructions for determining whether a packet is newlyseen. Additionally, the computer readable medium includes instructionsfor transmitting an acknowledgement (ACK), wherein a packet isacknowledged when the receiving node receives the linear combination ofpackets and determines which packet of the linear combination of packetshas been newly seen.

Yet another embodiment includes a computer readable medium havingcomputer readable code thereon for providing a network coding based flowcontrol at an intermediate node. The computer readable medium includesinstructions for receiving a linear combination of packets. The computerreadable medium also includes instructions for determining a linearcombination of packets to transmit from a transmit queue. The computerreadable medium further includes instructions for transmitting thelinear combination of packets across a network using a sliding windowprotocol.

Still other embodiments include a computerized device, configured toprocess all the method operations disclosed herein as embodiments of theinvention. In such embodiments, the computerized device includes amemory system, a processor, communications interface in aninterconnection mechanism connecting these components. The memory systemis encoded with a process that provides network coding based flowcontrol as explained herein that when performed (e.g. when executing) onthe processor, operates as explained herein within the computerizeddevice to perform all of the method embodiments and operations explainedherein as embodiments of the invention. Thus any computerized devicethat performs or is programmed to perform up processing explained hereinis an embodiment of the invention.

Other arrangements of embodiments of the invention that are disclosedherein include software programs to perform the method embodiment stepsand operations summarized above and disclosed in detail below. Moreparticularly, a computer program product is one embodiment that has acomputer-readable medium including computer program logic encodedthereon that when performed in a computerized device provides associatedoperations providing network coding based flow control as explainedherein. The computer program logic, when executed on at least oneprocessor with a computing system, causes the processor to perform theoperations (e.g., the methods) indicated herein as embodiments of theinvention. Such arrangements of the invention are typically provided assoftware, code and/or other data structures arranged or encoded on acomputer readable medium such as an optical medium (e.g., CD-ROM),floppy or hard disk or other a medium such as firmware or microcode inone or more ROM or RAM or PROM chips or as an Application SpecificIntegrated Circuit (ASIC) or as downloadable software images in one ormore modules, shared libraries, etc. The software or firmware or othersuch configurations can be installed onto a computerized device to causeone or more processors in the computerized device to perform thetechniques explained herein as embodiments of the invention. Softwareprocesses that operate in a collection of computerized devices, such asin a group of data communications devices or other entities can alsoprovide the system of the invention. The system of the invention can bedistributed between many software processes on several datacommunications devices, or all processes could run on a small set ofdedicated computers, or on one computer alone.

It is to be understood that the embodiments of the invention can beembodied strictly as a software program, as software and hardware, or ashardware and/or circuitry alone, such as within a data communicationsdevice.

Note that each of the different features, techniques, configurations,etc. discussed in this disclosure can be executed independently or incombination. Accordingly, the present invention can be embodied andviewed in many different ways. Also, note that this summary sectionherein does not specify every embodiment and/or incrementally novelaspect of the present disclosure or claimed invention. Instead, thissummary only provides a preliminary discussion of different embodimentsand corresponding points of novelty over conventional techniques. Foradditional details, elements, and/or possible perspectives(permutations) of the invention, the reader is directed to the DetailedDescription section and corresponding figures of the present disclosureas further discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing will be apparent from the following more particulardescription of preferred embodiments of the invention, as illustrated inthe accompanying drawings in which like reference characters refer tothe same parts throughout the different views. The drawings are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the invention.

FIG. 1 comprises a diagram showing seen packets and witnesses in termsof a basis matrix;

FIG. 2 depicts an example of coding and ACKs;

FIG. 3 depicts a protocol stack showing a new network coding layer;

FIG. 4 comprises a diagram of a simulation topology;

FIG. 5 comprises a diagram showing fairness and compatibility for oneTCP/NC flow and one TCP flow;

FIG. 6 comprises a diagram showing throughput versus redundancy forTCP/NC flows;

FIG. 7 comprises a diagram showing throughput versus loss rate for TCPand TCP/NC flows;

FIG. 8 comprises a diagram showing throughput with and withoutintermediate node re-coding;

FIG. 9 illustrates a particular embodiment a method for a transmittingnode to perform network coding based flow control in accordance withembodiments of the invention;

FIG. 10 illustrates a particular embodiment a method for a receivingnode to perform network coding based flow control in accordance withembodiments of the invention;

FIG. 11 illustrates a particular embodiment a method for an intermediatenode to perform network coding based flow control in accordance withembodiments of the invention;

FIG. 12 illustrates a high level block diagram of a coding buffer inaccordance with embodiments of the invention;

FIG. 13 is a diagram of a network coding header in accordance withembodiments of the invention;

FIG. 14 is a diagram showing receiver side window management inaccordance with embodiments of the invention;

FIG. 15 is a graph showing goodput versus redundancy factor inaccordance with embodiments of the invention;

FIG. 16 is a graph showing goodput versus coding window size inaccordance with embodiments of the invention; and

FIG. 17 is a graph showing goodput versus packet loss rate in accordancewith embodiments of the invention.

DETAILED DESCRIPTION

In order to bring the ideas of network coding into practice, a protocolis required that brings out the benefits of network coding whilerequiring very little change in the protocol stack. Flow control andcongestion control in today's Internet are predominantly based on theTransmission Control Protocol (TCP), which works using the idea of asliding transmission window of packets, whose size is controlled basedon feed-back. The TCP paradigm has clearly proven successful. It wouldtherefore be desirable to provide a sliding-window approach as similaras possible to TCP for network coding that makes use of acknowledgmentsfor flow and congestion control.

Such an approach would necessarily differ from the generation-basedapproach more commonly considered for network coding. Described below ishow to incorporate network coding into TCP, allowing its use withminimal changes to the protocol stack, and in such a way thatincremental deployment is possible.

The main idea behind TCP is to use acknowledgments of newly receivedpackets as they arrive in correct sequence order in order to guaranteereliable transport and also to provide a feedback signal for thecongestion control loop. This mechanism requires some modification forsystems using network coding. The key difference to be dealt with isthat under network coding the receiver does not obtain original packetsof the message, but linear combinations of the packets that are thendecoded to obtain the original message once enough such combinationshave arrived. Hence, the notion of an ordered sequence of packets asused by TCP is missing, and further, a linear combination may bring innew information to a receiver even though it may not reveal an originalpacket immediately. The current ACK mechanism does not allow thereceiver to acknowledge a packet before it has been decoded. For networkcoding, a modification of the standard TCP mechanism is required thatacknowledges every unit of information received.

A new unit of information corresponds mathematically to a degree offreedom; essentially, once n degrees of freedom have been obtained, amessage that would have required n unencoded packets can be decoded.Described herein is a mechanism that performs the functions of TCP,namely reliable transport and congestion control, based on acknowledgingevery degree of freedom received, whether or not it reveals a newpacket.

A new network coding layer is introduced between the transport layer andthe network layer of the protocol stack. Thus, the congestion controlprinciple of TCP is recycled, namely that the number of packets involvedin transmissions cannot exceed the number of acknowledgments received bymore than the congestion window size. However, two main changes areintroduced. First, whenever the source is allowed to transmit, it sendsa random linear combination of all packets in the congestion window.Second, the receiver acknowledges degrees of freedom and not originalpackets. An appropriate interpretation of the degree of freedom allowsone to order the receiver degrees of freedom in a manner consistent withthe packet order of the source. This allows the standard TCP protocol tobe utilized with the minimal change. In particular, a TCP-Vegas protocolis used, as this protocol is more compatible with the requiredmodifications.

The present technique does not rely on the link layer for recoveringlosses. Instead, an erasure correction scheme based on random linearcodes across packets is used. Coding across packets is a natural way tohandle losses. A coding based approach is better suited forbroadcast-mode opportunistic routing scenarios, as randomly chosenlinear combinations of packets are more likely to convey newinformation, compared to retransmissions. The MORE scheme explains thebenefits of network coding in the context of opportunistic routing.However, the problem with MORE is the batch processing that makes itless compatible to a sliding window protocol such as TCP. By providingan interface between TCP and a network coded system, a new approach ispresented to implementing TCP over wireless networks, and it is herewhere the benefits of the presently disclosed method and apparatus forproviding network coding based flow control are most dramatic.

It is important to note that the present scheme respects the end-to-endphilosophy of TCP—it would work even if coding operations were performedonly at the end hosts. Further, if some nodes inside the network alsoperform network coding, the present technique naturally generalizes tosuch scenarios as well.

Definitions are now introduced. Packets are treated as vectors over afinite field F_(q) of size q. All the discussion here is with respect toa single source that generates a stream of packets. The k^(th) packetthat the source generates is said to have an index k and is denoted asp_(k).

Definition 1 (Seeing a packet): A node is said to have seen a packetp_(k) if it has enough information to compute a linear combination ofthe form (p_(k)+q), where q=Σ_(l>k)α_(l)p_(l), with α_(l)ε

_(q) for all l>k. Thus, q is a linear combination involving packets withindices larger than k.

The notion of “seeing” a packet is a natural extension of the notion of“decoding” a packet, or more specifically, receiving a packet in thecontext of classical TCP. For example, if a packet is decoded then it isindeed also seen, with q=0. A node can compute any linear combinationwhose coefficient vector is in the span of the coefficient vectors ofpreviously received linear combinations. This leads to the followingdefinition.

Definition 2 (Knowledge of a node): The knowledge of a node is the setof all linear combinations of original packets that it can compute,based on the information it has received so far. The coefficient vectorsof these linear combinations form a vector space called the knowledgespace of the node.

Proposition 1: If a node has seen packet p_(k) then it knows exactly onelinear combination of the form p_(k)+q such that q is itself a linearcombination involving only unseen packets. The above propositioninspires the following definition.

Definition 3 (Witness): The unique linear combination guaranteed byProposition 1 is called the witness for seeing p_(k).

A compact representation of the knowledge space is the basis matrix.This is a matrix in row-reduced echelon form (RREF) such that its rowsform a basis of the knowledge space. FIG. 1 explains the notion of aseen packet in terms of the basis matrix. Essentially, the seen packetsare the ones that correspond to the pivot columns of the basis matrix.Given a seen packet, the corresponding pivot row gives the coefficientvector for the witness linear combination. An important observation isthat the number of seen packets is always equal to the dimension of theknowledge space, or the number of degrees of freedom that have beenreceived so far. A newly received linear combination that increases thedimension is said to be innovative. It is assumed that the field size isvery large. As a consequence, each reception will be innovative withhigh probability, and will cause the next unseen packet to be seen.

Example: Suppose a node knows the following linear combinations:x=(p₁+p₂) and y=(p₁+p₃). Since these are linearly independent, theknowledge space has a dimension of 2. Hence, the number of seen packetsmust be 2. It is clear that packet p₁ has been seen, since x satisfiesthe requirement of Definition 1. Now, the node can compute z

x−y=(p₂−p₃).

Thus, it has also seen p₂. That means p₃ is unseen. Hence, y is thewitness for p_(i), and z is the witness for p₂.

In this section, the logical description of the new protocol ispresented, followed by a way to implement these ideas with as littledisturbance as possible to the existing protocol stack.

One aim of the present algorithm is to mask losses from TCP using randomlinear coding. Some important modifications are made in order toincorporate coding. First, instead of the original packets, randomlinear combinations of packets in the congestion window are transmitted.While such coding helps with erasure correction, it also leads to aproblem in acknowledging data. TCP operates with units of packets, whichhave a well-defined ordering. Thus, the packet sequence number can beused for acknowledging the received data. The unit in the presentprotocol is a degree of freedom. However, when packets are codedtogether, there is no clear ordering of the degrees of freedom that canbe used for ACKs. The notion of seen packets defines an ordering of thedegrees of freedom that is consistent with the packet sequence numbers,and can therefore be used to acknowledge degrees of freedom.

Upon receiving a linear combination, the sink finds out which packet, ifany, has been newly seen because of the new arrival and acknowledgesthat packet. The sink thus pretends to have received the packet even ifit cannot be decoded yet. It will be shown below that at the end this isnot a problem because if all the packets in a file have been seen, thenthey can all be decoded as well.

The idea of transmitting random linear combinations and acknowledgingseen packets achieves the goal of masking losses from TCP as follows.With a large field size, every random linear combination is very likelyto cause the next unseen packet to be seen. So, even if a transmittedlinear combination is lost, the next successful reception will cause thenext unseen packet to be seen. From TCP's perspective, this appears asthough the degree of freedom waits in a fictitious queue until thechannel stops erasing packets and allows it through. Thus, there willnever be any duplicate ACKs. Every ACK will cause the congestion windowto advance. In short, the lossiness of the link is presented to TCP asan additional queuing delay that leads to a larger effective round-triptime. The term round-trip time thus has a new interpretation. It is theeffective time the network takes to reliably deliver a degree of freedom(including the delay for the coded redundancy, if necessary), followedby the return of the ACK. This is larger than the true network delay ittakes for a transmission and the return of the ACK. The more lossy thelink is, the larger will be the effective RTT. Presenting TCP with alarger value for RTT may seem counterintuitive as TCP's rate isinversely related to RTT. However, if done correctly, it improves therate by preventing loss-induced window closing, as it gives the networkmore time to deliver the data in spite of losses, before TCP times out.Therefore, losses are effectively masked.

Now discussed will be how the effectively masked losses affectcongestion control. Since losses are masked from the congestion controlalgorithm, the TCP-Reno style approach to congestion control usingpacket loss as a congestion indicator is not well suited to thissituation. However, it is useful to note that the congestion relatedlosses are made to appear as a longer RTT. Therefore, an approach isneeded that infers congestion from an increase in RTT. The naturalchoice is TCP-Vegas.

TCP-Vegas uses a proactive approach to congestion control by inferringthe size of the network buffers even before they start dropping packets.The crux of the algorithm is to estimate the round-trip time (RTT) anduse this information to find the discrepancy between the expected andactual transmission rate. As congestion arises, buffers start to fill upand the RTT starts to rise, and this is used as the congestion signal.This signal is used to adjust the congestion window and hence the rate.

In order to use TCP-Vegas correctly in this setting, the effective RTTof a degree of freedom needs to be used, including the fictitiousqueuing delay. In other words, the RTT should be measured from the pointwhen a packet is first sent out from TCP, to the point when the ACKreturns saying that this packet has been seen. This is indeed the casefor the default RTT measurement mechanism of TCP-Vegas. The TCP sendernotes down the transmission time of every packet. When an ACK arrives,it is matched to the corresponding transmit timestamp in order tocompute the RTT. Thus, no modification is required.

Consider the example shown in FIG. 2. Suppose the congestion window'slength is four. Assume TCP sends 4 packets to the network coding layerat t=0. All four transmissions are linear combinations of these fourpackets. The 1^(st) transmission causes the 1^(st) packet to be seen.The 2^(nd) and 3^(rd) transmissions are lost, and the 4^(th)transmission causes the 2^(nd) packet to be seen (the discrepancy isbecause of losses). As far as the RTT estimation is concerned,transmissions 2, 3 and 4 are treated as attempts to convey the 2^(nd)degree of freedom. The RTT for the 2^(nd) packet must include the finalattempt that successfully delivers the 2^(nd) degree of freedom, namelythe 4^(th) transmission. In other words, the RTT is the time from t=0until the reception of ACK=3.

The implementation of all these ideas in the existing protocol stackneeds to be done in as non-intrusive a manner as possible. A solution ispresented which embeds the network coding operations in a separate layerbelow TCP and above IP on the source and receiver side, as shown in FIG.3. The exact operation of these modules will now be described.

The sender module accepts packets from the TCP source and buffers theminto an encoding buffer which represents the coding window, until theyare ACKed by the receiver. The sender then generates and sends randomlinear combinations of the packets in the coding window. Thecoefficients used in the linear combination are also conveyed in theheader.

For every packet that arrives from TCP, R linear combinations are sentto the IP layer on average, where R is the redundancy parameter. Theaverage rate at which linear combinations are sent into the network isthus a constant factor more than the rate at which TCP's congestionwindow progresses. This is necessary in order to compensate for the lossrate of the channel and to match TCP's sending rate to the rate at whichdata is actually sent to the receiver. If there is too littleredundancy, then the data rate reaching the receiver will not match thesending rate because of the losses. This leads to a situation where thelosses are not effectively masked from the TCP layer. Hence, there arefrequent losses leading to a low throughput. On the other extreme, toomuch redundancy is also bad, since then the transmission rate becomeslimited by the rate of the code itself. Besides, sending too many linearcombinations can congest the network. The ideal level of redundancy isto keep R equal to the reciprocal of the probability of successfulreception. Thus, in practice the value of R should be dynamicallyadjusted by estimating the loss rate, possibly using the RTT estimates.

Upon receiving a linear combination, the receiver module first retrievesthe coding coefficients from the header and appends it to the basismatrix of its knowledge space. Then, it performs a Gaussian eliminationto find out which packet is newly seen so that this packet can be ACKed.The receive module also maintains a buffer of linear combinations ofpackets that have not been decoded yet. Upon decoding the packets, thereceiver delivers them to the TCP sink.

The algorithm is specified below using pseudo-code. This specificationassumes a one-way TCP flow.

1) Source side: The source side algorithm has to respond to two types ofevents—the arrival of a packet from the source TCP, and the arrival ofan ACK from the receiver via IP.

1. Set NUM to 0.

2. Wait state: If any of the following events occurs, respond asfollows; else, wait.

3. Packet arrives from TCP sender:

-   -   a) If the packet is a control packet used for connection        management, deliver it to the IP layer and return to wait state.    -   b) If packet is not already in the coding window, add it to the        coding window.    -   c) Set NUM=NUM+R. (R=redundancy factor)    -   d) Repeat the following └NUM┘ times:        -   i) Generate a random linear combination of the packets in            the coding window.        -   ii) Add the network coding header specifying the set of            packets in the coding window and the coefficients used for            the random linear combination.        -   iii) Deliver the packet to the IP layer.    -   e) Set NUM:=fractional part of NUM.    -   f) Return to the wait state.

4. ACK arrives from receiver: Remove the ACKed packet from the codingbuffer and hand over the ACK to the TCP sender.

2) Receiver side: On the receiver side, the algorithm again has torespond to two types of events: the arrival of a packet from the source,and the arrival of ACKs from the TCP sink.

1. Wait state: If any of the following events occurs, respond asfollows; else, wait.

2. ACK arrives from TCP sink: If the ACK is a control packet forconnection management, deliver it to the IP layer and return to the waitstate; else, ignore the ACK.

3. Packet arrives from source side:

-   -   a) Remove the network coding header and retrieve the coding        vector.    -   b) Add the coding vector as a new row to the existing coding        coefficient matrix, and perform Gaussian elimination to update        the set of seen packets.    -   c) Add the payload to the decoding buffer. Perform the        operations corresponding to the Gaussian elimination on the        buffer contents. If any packet gets decoded in the process,        deliver it to the TCP sink and remove it from the buffer.    -   d) Generate a new TCP ACK with sequence number equal to that of        the oldest unseen packet.

The present protocol guarantees reliable transfer of information. Inother words, every packet in the packet stream generated by theapplication at the source will be delivered eventually to theapplication at the sink. The acknowledgment mechanism ensures that thecoding module at the sender does not remove a packet from the codingwindow unless it has been ACKed, i.e., unless it has been seen by thesink. Thus, if all packets in a file have been seen, then the file canbe decoded at the sink.

Theorem 1: From a file of n packets, if every packet has been seen, thenevery packet can also be decoded.

Proof: If the sender knows a file of n packets, then the sender'sknowledge space is of dimension n. Every seen packet corresponds to anew dimension. Hence, if all n packets have been seen, then thereceiver's knowledge space is also of dimension n, in which case it mustbe the same as the sender's and all packets can be decoded.

In other words, seeing n different packets corresponds to having nlinearly independent equations in n unknowns. Hence, the unknowns can befound by solving the system of equations. At this point, the file can bedelivered to the TCP sink. In practice, one does not have to necessarilywait until the end of the file to decode all packets. Some of theunknowns can be found even along the way. In particular, whenever thenumber of equations received catches up with the number of unknownsinvolved, the unknowns can be found. Now, for every new equationreceived, the receiver sends an ACK. The congestion control algorithmuses the ACKs to control the injection of new unknowns into the codingwindow. Thus, the discrepancy between the number of equations and numberof unknowns does not tend to grow with time, and therefore will hit zerooften based on the channel conditions. As a consequence, the decodingbuffer will tend to be stable.

An interesting observation is that the arguments used to show thesoundness of our approach are quite general and can be extended to moregeneral scenarios such as random linear coding based multicast overarbitrary topologies.

The protocol described above is simulated using the Network Simulator(ns-2). The topology for the simulations is a tandem network consistingof 4 hops (hence 5 nodes), shown in FIG. 4. The source and sink nodesare at opposite ends of the chain. Two FTP applications want tocommunicate from the source to the sink. There is no limit on the filesize. They emit packets continuously till the end of the simulation.They either use TCP without coding or TCP with network coding (denotedTCP/NC). In this simulation, intermediate nodes do not re-encodepackets. All the links have a bandwidth of 1 Mbps, and a propagationdelay of 100 ms. The buffer size on the links is set at 200. The TCPreceive window size is set at 100 packets, and the packet size is 1,000bytes. The Vegas parameters are chosen to be α=28, β=30, γ=2.

By fairness, if two similar flows compete for the same link, they mustreceive an approximately equal share of the link bandwidth. In addition,this must not depend on the order in which the flows join the network.The fairness of TCP-Vegas is a well-studied problem. It is known that,depending on the values chosen for the α and β parameters, TCP-Vegascould be unfair to an existing connection when a new connection entersthe bottleneck link. In the present simulations, values of α and β arechosen that allow fair sharing of bandwidth when two TCP flows withoutour modification compete with each other, in order to evaluate theeffect of the modification on fairness. With the same α and β two casesare considered:

Case 1: The situation where a network coded TCP flow competes withanother flow running TCP without coding.

Case 2: The situation where two coded TCP flows compete with each other.

In both cases, the loss rate is set to 0% and the redundancy parameteris set to 1 for a fair comparison. In the first case, the TCP flowstarts first at t=0.5 s and the flow starts at 1000 s. The system issimulated for 2000 s. The current throughput is calculated at intervalsof 2.5 s. The evolution of the throughput over time is shown in FIG. 5.FIGS. 5-8 are based on Simulations. FIG. 5 shows that the effect ofintroducing the coding layer does not affect fairness. After the secondflow starts, the bandwidth gets redistributed fairly.

For case 2, the experiment is repeated with the same starting times, butthis time both flows are TCP/NC flows. The plot for this case isessentially identical to FIG. 5 (and hence is not shown here) because inthe absence of losses, TCP/NC behaves identically to TCP if the effectsof field size are ignored. Thus, coding can coexist with TCP in theabsence of losses, without affecting fairness. The new protocol indeedachieves a high throughput, especially in the presence of losses.

A. Throughput of the New Protocol Simulation Results

The simulation setup is identical to that used in the fairnesssimulations. The effect of the redundancy parameter on the throughput ofTCP/NC for a fixed loss rate of 5% is shown. The loss rate, refers tothe probability of a packet getting lost on each link. Both packets inthe forward direction as well as ACKs in the reverse direction aresubject to these losses. No re-encoding is allowed at the intermediatenodes. Hence, the overall probability of packet loss across 4 hops isgiven by 1−(1−0.05)⁴ which is roughly 19%. Hence the capacity is roughly0.81 Mbps, which when split fairly gives 0.405 Mbps per flow. Thesimulation time is 10000 s.

Two TCP/NC flows are allowed to compete on this network, both startingat 0.5 s. Their redundancy parameter is varied between 1 and 1.5. Thetheoretically optimum value is approximately 1/(1−0.19)≅1.23. FIG. 6shows the plot of the throughput for the two flows, as a function of theredundancy parameter R. It is clear from the plot that R plays animportant role in TCP/NC. The throughput peaks around R=1.25. The peakthroughput achieved is 0.397 Mbps, which is indeed close to the capacitycalculated above. In the same situation, when two TCP flows compete forthe network, the two flows see a throughput of 0.0062 and 0.0072 Mbpsrespectively. Thus, with the correct choice of R, the throughput for theflows in the case is very high compared to the TCP case. In fact, evenwith R=1, TCP/NC achieves about 0.011 Mbps for each flow improving onTCP by almost a factor of 2.

Next, the variation of throughput with loss rate for both TCP and TCP/NCwill be discussed. The simulation parameters are all the same as above.The loss rate of all links is kept at the same value, and this is variedfrom 0 to 20%. We compare two scenarios-two TCP flows competing witheach other, and two TCP/NC flows competing with each other. For theTCP/NC case, the redundancy parameter is set at the optimum valuecorresponding to each loss rate. FIG. 7 shows that throughput fallsrapidly as losses increase. However, TCP/NC is very robust to losses andreaches a throughput that is close to capacity. (If p is the loss rateon each link, then capacity is (1−p)⁴, which must then be splitequally.)

FIG. 8 shows the instantaneous throughput in a 642 second longsimulation of a tandem network with 3 hops (i.e., 4 nodes), whereerasure probabilities vary with time in some specified manner. The thirdhop is on average, the most erasure-prone link. The plots are shown fortraditional TCP, TCP/NC with coding only at the source, and TCP/NC withre-encoding at node 3 (just before the worst link). The operation of there-encoding node is very similar to that of the source—it collectsincoming linear combinations in a buffer, and transmits, on average,R_(int) random linear combinations of the buffer contents for everyincoming packet. The R of the sender is set at 1.8, and the R_(int) ofnode 3 is set at 1.5 for the case when it re-encodes. The averagethroughput is shown in the table. A considerable improvement is seen dueto the coding, that is further enhanced by allowing intermediate nodere-encoding. This plot thus shows that the present scheme is also suitedto systems with coding inside the network.

These simulations are meant to be a preliminary study of the presentalgorithm's performance. Specifically, the following points must benoted:

Link layer retransmission is not considered for either TCP or TCP/NC. Ifallowed, this could improve the performance of TCP. However, asmentioned earlier, the retransmission approach does not extend to moregeneral multipath routing solutions, whereas coding is better suited tosuch scenarios.

The throughput values do not account for the overhead associated withthe network coding headers. The main overhead is in conveying the codingcoefficients and the contents of the coding window. If the source andsink share a pseudorandom number generator, then the coding coefficientscan be conveyed succinctly by sending the current state of thegenerator. Similarly, the coding window contents can be conveyed in anincremental manner to reduce the overhead.

The loss in throughput due to the finiteness of the field has not beenmodeled in the simulations. A small field might cause received linearcombinations to be non-innovative, or might cause packets to be seen outof order, resulting in duplicate ACKs. However, the probability thatsuch problems persist for a long time falls rapidly with the field size.For practical choices of field size, these issues will only causetransient effects that will not have a significant impact onperformance.

In this section, an idealized scenario is discussed in order to providea first order analysis of the new protocol. The key ideas of the presentprotocol are explained with emphasis on the interaction between thecoding operation and the feedback. The model used in this section willalso serve as a platform to incorporate more practical situations. Thecongestion control aspect of the problem is abstracted out by assumingthat the capacity of the system is fixed in time and known at thesource, and hence the arrival rate is always maintained below thecapacity. It is also assumed that nodes have infinite capacity buffersto store packets. A topology that consists of a chain of erasure-pronelinks in tandem is discussed, with perfect end-to-end feedback from thesink directly to the source. In such a system, the behavior of the queuesizes at various nodes is discussed.

System model: The network studied in this section is a daisy chain of Nnodes, each node being connected to the next one by a packet erasurechannel. A slotted time system is assumed. The source generates packetsaccording to a Bernoulli process of rate λ packets per slot. The pointof transmission is at the very beginning of a slot. Just after thispoint, every node transmits one random linear combination of the packetsin its queue. Propagation delay is ignored. Thus, the transmission, ifnot erased by the channel, reaches the next node in the chain almostimmediately. However, the node may use the newly received packet only inthe next slot's transmission. Perfect, delay-free feedback from the sinkto the source is assumed. In every slot, the sink generates the feedbacksignal after the instant of reception of the previous node'stransmission. The erasure event happens with a probability (1−μ_(i)) onthe channel connecting node i and (i+1), and is assumed to beindependent across different channels and over time. Thus, the systemhas a capacity min_(i)μ_(i) packets per slot. A further assumption ismade that λ<min_(i)μ_(i), and the load factor is defined asp_(i)=λ/μ_(i). The relation between the transmitted linear combinationand the original packet stream is conveyed in the packet header. Thisoverhead is ignored for the analysis in this section.

This model and the following analysis also works for the case when notall intermediate nodes are involved in the network coding. If some nodesimply forwards the incoming packets, then this can be incorporated inthe following way. An erasure event on either the link entering thisnode or the link leaving this node will cause a packet erasure. Hence,these two links can be replaced by a single link whose probability ofbeing ON is simply the product of the ON probabilities of the two linksbeing replaced. Thus, all non-coding nodes can be removed from themodel, which brings one back to the same situation as in the abovemodel.

Queue update mechanism: Each node transmits a random linear combinationof the current contents of its queue and hence, the question of how toupdate the queue contents becomes important. In every slot, the sinksends an ACK directly to the source, containing the index of the oldestpacket not yet seen by the sink. Upon receiving the ACK, the sourcedrops all packets from its queue with an index lower than the sink'srequest. The intermediate nodes do not have direct feedback from thesink. Hence, the source has to inform them about the sink's ACK. Thisinformation is sent on the same erasure channel used for the regulartransmission. This feed-forward of the sink's status is modeled asfollows. Whenever the channel entering an intermediate node is in the ONstate (i.e., no erasure), the node's version of the sink's status isupdated to that of the previous node. In practice, the source need nottransmit the sink's status explicitly. The intermediate nodes can inferit from the set of packets that have been involved in the linearcombination—if a packet is no longer involved, that means the sourcemust have dropped it, implying that the sink must have ACKed it already.Whenever an intermediate node receives an innovative packet, this causesthe node to see a previously unseen packet. The node performs a Gaussianelimination to compute the witness of the newly seen packet, and addsthis to the queue. Thus, intermediate nodes store the witnesses of thepackets that they have seen. The queue update rule is similar to that ofthe source. An intermediate node drops the witness of all packets up tobut excluding the one requested by the sink. This is based on the mostupdated version of the sink's status known at the intermediate node.

Queuing analysis: The following theorem shows that if we allow coding atintermediate nodes, then it is possible to achieve the capacity of thenetwork, namely min_(i)μ_(i). Note that this theorem also implies thatif forwarding is only allowed at some of the intermediate nodes, thenthe capacity of a new network can still be derived by collapsing thelinks across the non-coding nodes.

Theorem 2: As long as λ<μ_(k) for all 0≦k<N, the queues at all the nodeswill be stable. The expected queue size in steady state at node k(0≦k<N) is given by:

${\left\lbrack Q_{k} \right\rbrack} = {{\sum\limits_{i = k}^{N - 1}\;\frac{\rho_{i}\left( {1 - \mu_{i}} \right)}{\left( {1 - \rho_{i}} \right)}} + {\sum\limits_{i = 1}^{k - 1}\;\rho_{i}}}$

An implication: Consider a case where all the p_(i)s are equal to somep. Then, the above relation implies that in the limit of heavy traffic,p→1, the queues are expected to be longer at nodes near the source thannear the sink.

A useful lemma: The following lemma shows that the random linear codingscheme has the property that every time there is a successful receptionat a node, the node sees the next unseen packet with high probability,provided the field is large enough. This fact will prove useful whileanalyzing the evolution of the queues.

Lemma 1: Let S_(A) and S_(B) be the set of packets seen by two nodes Aand B respectively. Assume S_(A)\S_(B) is non-empty. Suppose A sends arandom linear combination of its witnesses of packets in S_(A) and Breceives it successfully. The probability that this transmission causesB to see the oldest packet in S_(A)\S_(B) is

$\left( {1 - \frac{1}{q}} \right),$where q is the field size.

Proof: Let M_(A) be the RREF basis matrix for A. Then, the coefficientvector of the linear combination sent by A is t=uM_(A) where u is avector of length |S_(A)| whose entries are independent and uniformlydistributed over the finite field

F_(q). Let d* denote the index of the oldest packet in S_(A)\S_(B).

Let M_(B) be the RREF basis matrix for B before the new reception.Suppose t is successfully received by B. Then, B will append t as a newrow to M_(B) and perform Gaussian elimination. The first step involvessubtracting from t, suitably scaled versions of the pivot rows such thatall entries of t corresponding to pivot columns of M_(B) become 0. Weneed to find the probability that after this step, the leading non-zeroentry occurs in column d*, which corresponds to the event that B seespacket d*. Subsequent steps in the Gaussian elimination will not affectthis event. Hence, we focus on the first step.

Let P_(B) denote the set of indices of pivot columns of M_(B). In thefirst step, the entry in column d* of t becomes

${t^{\prime}\left( d^{*} \right)} = {{t\left( d^{*} \right)} - {\sum\limits_{{i \in P_{B}},{i < d^{*}}}^{\;}\;{{t(i)} \cdot {M_{B}\left( {{r_{B}(i)},d^{*}} \right)}}}}$

where r_(B)(i) is the index of the pivot row corresponding to pivotcolumn i in M_(B). Now, due to the way RREF is defined,t(d*)=u(r _(A)(d*)),

where r_(A)(i) denotes the index of the pivot row corresponding to pivotcolumn i in M_(A). Thus, t(d*) is uniformly distributed. Also, for i<d*,t(i) is a function of only those u(j)'s such that j<r_(A)(d*). Hence,t(d*) is independent of t(d*) for i<d*. From these observations and theabove expression for t′(d*), it follows that for any given M_(A) andM_(B), t′(d*) has a uniform distribution over

_(q), and the probability that it is not zero is therefore

$\left( {1 - \frac{1}{q}} \right).$

For the queuing analysis, it is assumed that a successful receptionalways causes the receiver to see its next unseen packet, provided thetransmitter has already seen it. A consequence of this assumption isthat the set of packets seen by a node is always a contiguous set, withno gaps in between. In particular, there is no repeated ACK due topackets being seen out of order. The above lemma argues that theseassumptions become more and more valid as the field size increases. Inreality, some packets may be seen out of order resulting in larger queuesizes. However, this effect is minor and can be neglected for a firstorder analysis.

The expected queue size: arrival and departure are defined as follows. Apacket is said to arrive at a node when the node sees the packet for thefirst time. A packet is said to depart from the node when the node dropsthe witness of that packet from its queue. For each intermediate node,we now study the expected time between the arrival and departure of anarbitrary packet at that node as this is related to the expected queuesize at the node, by Little's law.

Proof Theorem 2: Consider the k^(th) node, for 1≦k<N. The time a packetspends in this node's queue has two parts:

1) Time until the packet is seen by the sink:

The difference between the number of packets seen by a node and thenumber of packets seen by the next node downstream essentially behaveslike a Geom/Geom/1 queue. The Markov chain governing this evolution isidentical to that of the virtual queues. Given that a node has seen thepacket, the time it takes for the next node to see that packetcorresponds to the waiting time in a virtual queue. For a load factor ofp and a channel ON probability of μ the expected waiting time wasderived to be

$\frac{\left( {1 - \mu} \right)}{\mu\left( {1 - \rho} \right)}.$Now, the expected time until the sees the packet is the sum of (N−k)such terms, which gives

$\sum\limits_{i = k}^{N - 1}\;{\frac{\left( {1 - \mu_{i}} \right)}{\mu_{i}\left( {1 - \rho_{i}} \right)}.}$

2) Time until sink's ACK reaches intermediate node:

The ACK informs the source that the sink has seen the packet. Thisinformation needs to reach node k by the forward mechanism. The expectedtime for this information to move from node i to node i+1 is theexpected time until the next slot when the channel is ON, which is just

$\frac{1}{\mu_{i}}$(since the i^(th) channel is ON with probability μ_(l)). Thus, the timeit takes for the sink's ACK to reach node k is given by

$\sum\limits_{i = k}^{k - 1}\frac{1}{\mu_{i}}$

The total expected time T_(k) a packet spends in the queue at the k^(th)node (1≦k<N) can thus be computed by adding the above two terms. Now,assuming the system is stable (i.e., λ<min_(i)μ_(l)), Little's law canbe used to derive the expected queue size at the k^(th) node, bymultiplying T_(k) by λ:

${\left\lbrack Q_{k} \right\rbrack} = {{\sum\limits_{i = k}^{N - 1}\;\frac{\rho_{i}\left( {1 - \mu_{i}} \right)}{\left( {1 - \rho_{i}} \right)}} + {\sum\limits_{i = 1}^{k - 1}\;\rho_{i}}}$

A salient feature of the present technique is that it is simultaneouslycompatible with the case where only end hosts perform coding (therebypreserving the end-to-end philosophy of TCP), as well as the case whereintermediate nodes perform network coding. Theory suggests that a lotcan be gained by allowing intermediate nodes to code as well.Simulations show that the proposed changes lead to large throughputgains over TCP in lossy links, even with coding only at the source. Forinstance, in a tandem network with a 5% loss rate on each link, thethroughput goes up from about 0.007 Mbps to about 0.39 Mbps for thecorrect redundancy factor. Intermediate node coding further increasesthe gains.

Flow diagrams of particular embodiments of the presently disclosedmethods are depicted in FIGS. 9-11. The rectangular elements are hereindenoted “processing blocks” and represent computer software instructionsor groups of instructions. Alternatively, the processing blocksrepresent steps performed by functionally equivalent circuits such as adigital signal processor circuit or an application specific integratedcircuit (ASIC). The flow diagrams do not depict the syntax of anyparticular programming language. Rather, the flow diagrams illustratethe functional information one of ordinary skill in the art requires tofabricate circuits or to generate computer software to perform theprocessing required in accordance with the present invention. It shouldbe noted that many routine program elements, such as initialization ofloops and variables and the use of temporary variables are not shown. Itwill be appreciated by those of ordinary skill in the art that unlessotherwise indicated herein, the particular sequence of steps describedis illustrative only and can be varied without departing from the spiritof the invention. Thus, unless otherwise stated the steps describedbelow are unordered meaning that, when possible, the steps can beperformed in any convenient or desirable order.

Referring now to FIG. 9, a particular embodiment of a method 100 ofproviding network coding based flow control by way of a transmittingnode is shown. Method 100 begins with processing block 102 whichdiscloses determining a linear combination of packets to transmit from atransmit queue. As shown in processing block 104, the linear combinationof packets is linearly independent of previously transmitted linearcombinations of packets and therefore conveys new information.

Processing block 106 states transmitting the linear combination ofpackets across a network using a sliding window protocol. As furtherrecited in processing block 108, the transmitting the linear combinationof packets across a network using a sliding window protocol furthercomprises including coding coefficients in a header of the linearcombination of packets. These coding coefficients are used in decodingthe packet by the receiving node. Processing block 110 recites whereinthe network comprises a network operating in accordance with aTransmission Control Protocol (TCP).

Processing block 112 discloses receiving an acknowledgement (ACK),wherein a packet is acknowledged when a receiving node receives thelinear combination of packets and determines which packet of the linearcombination of packets has been newly seen. As shown in processing block114, and described in detail below, in a particular embodiment thereceiving an ACK comprises receiving a request of a byte sequence numberof a first byte of a first unseen packet. Processing block 116 statesremoving a packet from the transmit queue which has been acknowledged.

Referring now to FIG. 10, a particular embodiment of a method 150 ofproviding network coding based flow control by way of a receiving nodeis shown. Method 150 begins with processing block 152 which disclosesreceiving a linear combination of packets across a network using asliding window protocol. As shown in processing block 154, the networkcomprises a network operating in accordance with a Transmission ControlProtocol (TCP).

Processing block 156 states determining whether a packet is newly seen.As further recited in processing block 158, determining whether a packetis newly seen includes retrieving coding coefficients from a header ofthe linear combination of packets, appending the coding coefficients toa basis matrix, and performing a Gaussian elimination to determine whichpacket is newly seen so that the newly seen packet can be acknowledged.

Processing continues with processing block 160 which states transmittingan acknowledgement (ACK), wherein a packet is acknowledged when thereceiving node receives the linear combination of packets and determineswhich packet of the linear combination of packets has been newly seen.As shown in processing block 162, and described in detail below, in aparticular embodiment the receiving an ACK comprises receiving a requestof a byte sequence number of a first byte of a first unseen packet.

Processing block 164 recites maintaining a buffer of linear combinationsof packets that have not been decoded yet. Processing block 166discloses delivering decoded packets to a node.

Referring now to FIG. 11, a particular embodiment of a method 200 ofproviding network coding based flow control by way of an intermediatenode is shown. Method 200 begins with processing block 202 whichdiscloses receiving a linear combination of packets. Processing block204 states determining a linear combination of packets to transmit froma transmit queue.

Processing continues with processing block 206 which recitestransmitting the linear combination of packets across a network using asliding window protocol. As shown in processing block 208, the networkcomprises a network operating in accordance with a Transmission ControlProtocol (TCP).

Processing block 210 discloses determining if a packet is no longerinvolved in received linear combinations of packets, and removing thepacket from the transmit queue. Processing block 212 states when theintermediate node receives an innovative packet, determining from theinnovative packet a newly seen packet and adding the newly seen packetto the transmit queue.

Described above is a new protocol called TCP/NC that incorporatesnetwork coding inside the TCP/IP protocol stack with the aim ofimproving TCP throughput in wireless networks. The interface of TCP withnetwork coding can be viewed as a generalization of previous workcombining TCP with Forward Erasure Correction (FEC) schemes. As opposedto coding only at the source, the protocol also allows intermediatenodes in the network to perform re-encoding of data. It is thus moregeneral than end-to-end erasure correction over a single path, and cantherefore, in principle, be used in multipath and multicast scenarios toobtain throughput benefits.

A real-life network coding implementation is now described, based on themechanism proposed earlier. In particular, explained are:

1) How to address the practical problems that arise in making thenetwork coding and decoding operations compatible with TCP's windowmanagement system, such as variable packet length, buffer management,and network coding overhead.

2) The compatibility of the present protocol with the widely used TCPReno; the earlier methodology considered only TCP Vegas.

3) Experimental results on the throughput benefits of the new protocolfor a TCP connection over a single-hop wireless link.

Before beginning, the implications of this new protocol for improvingthroughput in wireless networks are described. There has been a growinginterest in approaches that make active use of the intrinsic broadcastnature of the wireless medium. In the technique known as opportunisticrouting, a node broadcasts its packet, and if one of its neighborsreceives the packet, that node will forward the packet downstream,thereby obtaining a diversity benefit. If more than one of the neighborsreceives the packet, they will have to coordinate and decide who willforward the packet.

The MORE protocol proposed the use of intra-flow network coding incombination with opportunistic routing. The random linear mixing(coding) of incoming packets at a node before forwarding them downstreamwas shown to reduce the coordination overhead associated withopportunistic routing. Another advantage is that the coding operationcan be easily tuned to add redundancy to the packet stream to combaterasures. Such schemes can potentially achieve capacity for a multicastconnection.

Typical implementations use batches of packets instead of slidingwindows, and are generally therefore not compatible with TCP. ExOR usesbatching to reduce the coordination overhead, but this interacts badlywith TCP's window mechanism. MORE uses batching to perform the codingoperation. In this case, the receiver cannot acknowledge the packetsuntil an entire batch has arrived and has been successfully decoded.Since TCP performance heavily relies on the timely return of ACKs, sucha delay in the ACKs would affect the round-trip time calculation andthereby reduce the throughput.

Opportunistic routing also leads to reordering of packets, which isknown to interact badly with TCP, as reordering can cause duplicateACKs, and TCP interprets duplicate ACKs as a sign of congestion. Theearlier described technique proposes a TCP-compatible sliding windowcoding scheme in combination with a new acknowledgment mechanism forrunning TCP over a network coded system. The sender would transmit arandom linear combination of packets in the TCP congestion window. Thenew type of ACK allows the receiver to acknowledge every linearcombination (degree of freedom) that is linearly independent from thepreviously received linear combinations. The receiver does not have towait to decode a packet, but can send a TCP ACK for every degree offreedom received, thus eliminating the problems of using batchwise ACKs.

It was shown that if the linear combination happens over a large enoughfinite field, then every incoming random linear combination will, withhigh probability, generate a TCP ACK for the very next unacknowledgedpacket in order. This is because the random combinations do not have anyinherent ordering. The argument holds true even when multiple pathsdeliver the random linear combinations. Hence the use of random linearcoding with the acknowledgment of degrees of freedom can potentiallyaddress the TCP reordering problem for multipath opportunistic routingschemes. By presenting an implementation of the TCP/NC protocol, thiswork provides a way of combining TCP with network-coding-based multipathopportunistic routing protocols such as MORE.

The above description of the protocol assumes a fixed packet length,which allows all coding and decoding operations to be performedsymbol-wise on the whole packet. That is, an entire packet serves as thebasic unit of data (i.e., as a single unknown), with the implicitunderstanding that the exact same operation is being performed on everysymbol within the packet. The main advantage of this view is that thedecoding matrix operations (i.e., Gaussian elimination) can be performedat the granularity of packets instead of individual symbols. Also, theACKs are then able to be represented in terms of packet numbers.Finally, the coding vectors then have one coefficient for every packet,not every symbol. Note that the same protocol and analysis holds even ifthe basic unit of data is fixed as a symbol instead of a packet. Theproblem is that the complexity will be very high as the size of thecoding matrix will be related to the number of symbols in the codingbuffer, which is much more than the number of packets (typically, asymbol is one byte long).

In actual practice, TCP is a byte-stream oriented protocol in which ACKsare in terms of byte sequence numbers. If all packets are of fixedlength, one can still apply the packet-level approach, since there is aclear and consistent map between packet sequence numbers and bytesequence numbers. In reality, however, TCP might generate segments ofdifferent sizes. The choice of how many bytes to group into a segment isusually made based on the Maximum Transmission Unit (MTU) of thenetwork, which could vary with time. A more common occurrence is thatapplications may use the PUSH flag option asking TCP to packetize thecurrently outstanding bytes into a segment, even if it does not form asegment of the maximum allowed size. In short, it is important to ensurethat the protocol works correctly in spite of variable packet sizes.

A closely related problem is that of repacketization, which refers tothe situation where a set of bytes that were assigned to two differentsegments earlier by TCP may later be reassigned to the same segmentduring retransmission. As a result, the grouping of bytes into packetsunder TCP may not be fixed over time.

Both variable packet lengths and repacketization need to be addressedwhen implementing the coding protocol. To solve the first problem, ifthere are packets of different lengths, one could elongate the shorterpackets by appending sufficiently many dummy zero symbols until allpackets have the same length. This will work correctly as long as thereceiver is somehow informed how many zeros were appended to eachpacket. While transmitting these extra dummy symbols will decrease thethroughput, generally this loss will not be significant, as packetlengths are usually consistent.

However, if there is repacketization, then there is another problem,namely it is no longer possible to view a packet as a single unknown.This is because one would not have a one-to-one mapping between packetssequence numbers and byte sequence numbers; the same bytes may now occurin more than one packet. Repacketization appears to destroy theconvenience of performing coding and decoding at the packet level.

To counter these problems, the following solution is proposed. Thecoding operation involves the sender storing the packets generated bythe TCP source in a coding buffer. Any incoming TCP segment ispre-processed before adding it to the coding buffer as follows:

1) First, any part of the incoming segment that is already in the bufferis removed from the segment.

2) Next, a separate TCP packet is created out of each remainingcontiguous part of the segment.

3) The source and destination port information is removed. It will beadded later in the network coding header.

4) The packets are appended with sufficiently many dummy zero bytes, tomake them as long as the longest packet currently in the buffer. Everyresulting packet is then added to the buffer. This processing ensuresthat the packets in the buffer will correspond to disjoint andcontiguous sets of bytes from the byte stream, thereby restoring theone-to-one correspondence between the packet numbers and the bytesequence numbers. The reason the port information is excluded from thecoding is because port information is necessary for the receiver toidentify which TCP connection a coded packet corresponds to. Hence, theport information should not be involved in the coding. The remainingparts of the header are referred to as the TCP subheader.

Upon decoding the packet, the receiver can find out how many bytes arereal and how many are dummy using the Start_(i) and End_(i) headerfields in the network coding header (described below). With these fixesin place, the packet-level algorithm is ready to be used. All operationsare performed on the packets in the coding buffer. FIG. 12 shows atypical state of the buffer after this pre-processing. The gaps at theend of the packets correspond to the appended zeros. It is important tonote that the TCP control packets such as SYN packet and reset packetare allowed to bypass the coding buffer and are directly delivered tothe receiver without any coding.

A coded packet is created by forming a random linear combination of asubset of the packets in the coding buffer. The coding operations aredone over a field of size 256 in this implementation. In this case, afield symbol corresponds to one byte. The header of a coded packetshould contain information that the receiver can use to identify what isthe linear combination corresponding to the packet. The header structurewill now be discussed in more detail.

The network coding header has the structure shown in FIG. 13. Thetypical sizes (in bytes) of the various fields are written above them.The meaning of the various fields are described next:

Source and destination port: The port information is needed for thereceiver to identify the coded packet's session. It must not be includedin the coding operation. It is taken out of the TCP header and includedin the network coding header.

Base: The TCP byte sequence number of the first byte that has not beenACKed. The field is used by intermediate nodes or the decoder to decidewhich packets can be safely dropped from their buffers without affectingreliability.

n: The number of packets involved in the linear combination.

Start_(i): The starting byte of the ith packet involved in the linearcombination.

End_(i): The last byte of the ith packet involved in the linearcombination.

αi: The coefficient used for the ith packet involved in the linearcombination.

The Start_(i) (except Start₁) and End_(i) are expressed relative to theprevious packet's End and Start respectively, to save header space. Asshown in FIG. 12, this header format will add 5n+7 bytes of overhead forthe network coding header in addition to the TCP header, where n is thenumber of packets involved in a linear combination. (Note that the portinformation is not counted in this overhead, since it has been removedfrom the TCP header.) It may be possible to reduce this overhead byfurther optimizing the header structure.

In the theoretical version of the algorithm, the sender transmits arandom linear combination of all packets in the coding buffer. However,as noted above, the size of the header scales with the number of packetsinvolved in the linear combination. Therefore, mixing all packetscurrently in the buffer will lead to a very large coding header.

To solve this problem, only a constant-sized subset of the packetschosen from within the coding buffer are mixed. We call this subset thecoding window. The coding window evolves as follows. The algorithm usesa fixed parameter for the maximum coding window size W. The codingwindow contains the packet that arrived most recently from TCP (whichcould be a retransmission), and the (W−1) packets before it in sequencenumber, if possible. However, if some of the (W−1) preceding packetshave already been dropped, then the window is allowed to extend beyondthe most recently arrived packet until it includes W packets.

Note that this limit on the coding window implies that the code is nowrestricted in its power to correct erasures and to combatreordering-related issues. The choice of W will thus play an importantrole in the performance of the scheme. The correct value for W willdepend on the length of burst errors that the channel is expected toproduce. Other factors to be considered while choosing W are discussedbelow.

A packet is removed from the coding buffer if a TCP ACK has arrivedrequesting a byte beyond the last byte of that packet. If a new TCPsegment arrives when the coding buffer is full, then the segment withthe newest set of bytes must be dropped. This may not always be thenewly arrived segment, for instance, in the case of a TCP retransmissionof a previously dropped segment.

The decoder module's operations are outlined below. The main datastructure involved is the decoding matrix, which stores the coefficientvectors corresponding to the linear combinations currently in thedecoding buffer.

The receiver side module stores the incoming linear combination in thedecoding buffer. Then it unwraps the coding header and appends the newcoefficient vector to the decoding matrix. Gaussian elimination isperformed and the packet is dropped if it is not innovative (i.e. if itis not linearly independent of previously received linear combinations).After Gaussian elimination, the oldest unseen packet is identified.Instead of acknowledging the packet number a, the decoder acknowledgesthe last seen packet by requesting the byte sequence number of the firstbyte of the first unseen packet, using a regular TCP ACK. Note that thiscould happen before the packet is decoded and delivered to the receiverTCP. The port and IP address information for sending this ACK may beobtained from the SYN packet at the beginning of the connection. AnyACKs generated by the receiver TCP are not sent to the sender. They areinstead used to update the receive window field that is used in the TCPACKs generated by the decoder (see subsection below). They are also usedto keep track of which bytes have been delivered, for buffer management.

The Gaussian elimination operations are performed not only on thedecoding coefficient matrix, but correspondingly also on the codedpackets themselves. When a new packet is decoded, any dummy zero symbolsthat were added by the encoder are pruned using the coding headerinformation. A new TCP packet is created with the newly decoded data andthe appropriate TCP header fields and this is then delivered to thereceiver TCP.

The decoding buffer needs to store packets that have not yet beendecoded and delivered to the TCP receiver. Delivery can be confirmedusing the receiver TCP's ACKs. In addition, the buffer also needs tostore those packets that have been delivered but have not yet beendropped by the encoder from the coding buffer. This is because, suchpackets may still be involved in incoming linear combinations. The Basefield in the coding header addresses this issue. Base is the oldest bytein the coding buffer. Therefore, the decoder can drop a packet if itslast byte is smaller than Base, and in addition, has been delivered toand ACKed by the receiver TCP. Whenever a new linear combinationarrives, the value of Base is updated from the header, and any packetsthat can be dropped are dropped.

The buffer management can be understood using FIG. 14. It shows thereceiver side windows in a typical situation. In this case, Base is lessthan the last delivered byte. Hence, some delivered packets have not yetbeen dropped. There could also be a case where Base is beyond the lastdelivered byte, possibly because nothing has been decoded in a while.

The TCP receive window header field is used by the receiver to informthe sender how many bytes it can accept. Since the receiver TCP's ACKsare suppressed, the decoder must copy this information in the ACKs thatit sends to the sender. However, to ensure correctness, the value of theTCP receive window is modified based on the decoding buffer size. Thelast acceptable byte should thus be the minimum of the receiver TCP'slast acceptable byte and the last byte that the decoding buffer canaccommodate. Note that while calculating the space left in the decodingbuffer, the space occupied by data that has already been delivered tothe receiver is included because such data will get dropped when Base isupdated. If window scaling option is used by TCP, this needs to be notedfrom the SYN packet, so that the modified value of the receive windowcan be correctly reported. Ideally, a large enough decoding buffer sizeis chosen so that the decoding buffer would not be the bottleneck andthis modification would never be needed.

The choice of redundancy factor is based on the effective lossprobability on the links. For a loss rate of p_(e), with an infinitewindow W and using TCP Vegas, the theoretically optimal value of R is1/(1−p_(e)). The basic idea is that of the coded packets that are sentinto the network, only a fraction (1−p_(e)) of them are delivered onaverage. Hence, the value of R must be chosen so that in spite of theselosses, the receiver is able to collect linear equations at the samerate as the rate at which the unknown packets are mixed in them by theencoder. As discussed below, in practice, the value of R may depend onthe coding window size W. As W decreases, the erasure correctioncapability of the code goes down. Hence, we may need a larger R tocompensate and ensure that the losses are still masked from TCP. Anotherfactor that affects the choice of R is the use of TCP Reno. The TCP Renomechanism causes the transmission rate to fluctuate around the linkcapacity, and this leads to some additional losses over and above thelink losses. Therefore, the optimal choice of R may be higher than1/(1−p_(e)).

There are several considerations to keep in mind while choosing W, thecoding window size. The main idea behind coding is to mask the losses onthe channel from TCP. In other words, it is desirable to correct losseswithout relying on the ACKs. Consider a case where W is just 1. Then,this is a simple repetition code. Every packet is repeated R times onaverage. Now, such a repetition would be useful only for recovering onepacket, if it was lost. Instead, if W was say 3, then every linearcombination would be useful to recover any of the three packetsinvolved. Ideally, the linear combinations generated should be able tocorrect the loss of any of the packets that have not yet been ACKed. Forthis, W needs to be large. This may be difficult, since a large W wouldlead to a large coding header. Another penalty of choosing a large valueof W is related to the interaction with TCP Reno. This is discussedbelow.

The penalty of keeping W small on the other hand, is that it reduces theerror correction capability of the code. For a loss probability of 10%,the theoretical value of R is around 1.1. However, this assumes that alllinear combinations are useful to correct any packet's loss. Therestriction on W means that a coded packet can be used only forrecovering those W packets that have been mixed to form that codedpacket. In particular, if there is a contiguous burst of losses thatresult in a situation where the receiver has received no linearcombination involving a particular original packet, then that packetwill show up as a loss to TCP. This could happen even if the value of Ris chosen according to the theoretical value. To compensate, a larger Ris chosen.

The connection between W, R and the losses that are visible to TCP canbe visualized as follows. Imagine a process in which whenever thereceiver receives an innovative linear combination, one imaginary tokenis generated, and whenever the sender slides the coding window forwardby one packet, one token is used up. If the sender slides the codingwindow forward when there are no tokens left, then this leads to apacket loss that will be visible to TCP. The reason is, when thishappens, the decoder will not be able to see the very next unseen packetin order. Instead, it will skip one packet in the sequence. This willmake the decoder generate duplicate ACKs requesting that lost (i.e.,unseen) packet, thereby causing the sender to notice the loss.

In this process, W corresponds to the initial number of tokens availableat the sender. Thus, when the difference between the number of redundantpackets (linear equations) received and the number of original packets(unknowns) involved in the coding up to that point is less than W, thelosses will be masked from TCP. However, if this difference exceeds W,the losses will no longer be masked. A theoretically optimal value of Wis not known. However, it is expected that the value should be afunction of the loss probability of the link. For the experiment, valuesof W were chosen based on trial and error.

By adding enough redundancy, the coding operation essentially convertsthe lossiness of the channel into an extension of the round-trip time(RTT). This is why the earlier described technique proposed the use ofthe idea with TCP Vegas, since TCP Vegas controls the congestion windowin a smoother manner using RTT, compared to the more abrupt loss-basedvariations of TCP Reno. However, the coding mechanism is also compatiblewith TCP Reno. The choice of W plays an important role in ensuring thiscompatibility. The choice of W controls the power of the underlyingcode, and hence determines when losses are visible to TCP. Losses willbe masked from TCP as long as the number of received equations is nomore than W short of the number of unknowns involved in them. Forcompatibility with Reno, one needs to make sure that whenever thesending rate exceeds the link capacity, the resulting queue drops arevisible to TCP as losses. A very large value of W is likely to mask eventhese congestion losses, thereby temporarily giving TCP a false estimateof capacity. This will eventually lead to a timeout, and will affectthroughput. The value of W should therefore be large enough to mask thelink losses and small enough to allow TCP to see the queue drops due tocongestion.

It is important to implement the encoding and decoding operationsefficiently, since any time spent in these operations will affect theround-trip time perceived by TCP. The finite field operations overGF(256) have been optimized through the use of logarithms to multiplyelements. Over GF(256), each symbol is one byte long. Addition inGF(256) can be implemented easily as a bitwise XOR of the two bytes.

The main computational overhead on the encoder side is the formation ofthe random linear combinations of the buffered packets. The managementof the buffer also requires some computation, but this is small comparedto the random linear coding, since the coding has to be done on everybyte of the packets. Typically, packets have a length L of around 1500bytes. For every linear combination that is created, the codingoperation involves LW multiplications and L(W−1) additions over GF(256),where W is the coding window size. Note that this has to be done R timeson average for every packet generated by TCP. Since the coded packetsare newly created, allocating memory for them could also take time.

On the decoder side, the main operation is the Gaussian elimination.Note that, to identify whether an incoming linear combination isinnovative or not, we need to perform Gaussian elimination only on thedecoding matrix, and not on the coded packet. If it is innovative, thenwe perform the row transformation operations of Gaussian elimination onthe coded packet as well. This requires O(LW) multiplications andadditions to zero out the pivot columns in the newly added row. Thecomplexity of the next step of zeroing out the newly formed pivot columnin the existing rows of the decoding matrix vary depending on thecurrent size and structure of the matrix. Upon decoding a new packet, itneeds to be packaged as a TCP packet and delivered to the receiver.Since this requires allocating space for a new packet, this could alsobe expensive in terms of time.

As we will see in the next section, the benefits brought by the erasurecorrection begin to outweigh the overhead of the computation and codingheader for loss rates of about three percent. This could be improvedfurther by more efficient implementation of the encoding and decodingoperations.

The TCP/NC protocol requires no modification in the basic features ofthe TCP protocol on either the sender side or the receiver side.However, other special features of TCP that make use of the ACKs in waysother than to report the next required byte sequence number, will needto be handled carefully. For instance, implementing the timestamp optionin the presence of network coding across packets may require somethought. With TCP/NC, the receiver may send an ACK for a packet evenbefore it is decoded. Thus, the receiver may not have access to thetimestamp of the packet when it sends the ACK. Similarly, the TCPchecksum field has to be dealt with carefully. Since a TCP packet isACKed even before it is decoded, its checksum cannot be tested beforeACKing. One solution is to implement a separate checksum at the networkcoding layer to detect errors. In the same way, the various other TCPoptions that are available have to be implemented with care to ensurethat they are not affected by the premature ACKs.

The protocol was tested on a TCP flow running over a single-hop wirelesslink. The transmitter and receiver are Linux machines equipped with awireless antenna. The experiment is performed over 802.11a with abit-rate of 6 Mbps and a maximum of 5 link layer retransmissionattempts. RTS-CTS is disabled. The present implementation uses the Clickmodular router. In order to control the parameters of the setup, thepredefined elements of Click are used. Since the two machines arephysically close to each other, there are very few losses on thewireless link. Instead, packet losses were artificially induced usingthe RandomSample element. Note that these packet losses are introducedbefore the wireless link. Hence, they will not be recovered by the linklayer retransmissions, and have to be corrected by the layer above IP.The round-trip delay is empirically observed to be in the range of a fewtens of milliseconds. The encoder and decoder queue sizes are set to 100packets, and the size of the bottleneck queue just in front of thewireless link is set to 5 packets. In the setup, the loss inducingelement is placed before the bottleneck queue. The quantity measuredduring the experiment is the goodput over a 20 second long TCP session.The goodput is measured using iper f. Each point in the plots shown isaveraged over 4 or more iterations of such sessions, depending on thevariability. Occasionally, when the iteration does not terminate and theconnection times out, the corresponding iteration is neglected in theaverage, for both TCP and TCP/NC. This happens around 2% of the time,and is observed to be because of an unusually long burst of losses inthe forward or return path. In the comparison, neither TCP nor TCP/NCuses selective ACKs. TCP uses delayed ACKs. However, we have notimplemented delayed ACKs in TCP/NC at this point. FIGS. 15-17 are basedon real experiments. FIG. 15 shows the variation of the goodput withthe redundancy factor R for a loss rate of 10%, with a fixed codingwindow size of W=3. The theoretically optimal value of R for this lossrate is 1.11 (=1/0.9). However, from the experiment, the best goodput isachieved for an R of around 1.25. The discrepancy is possibly because ofthe type of coding scheme employed. The coding scheme transmits a linearcombination of only the W most recent arrivals, in order to save packetheader space. This restriction reduces the strength of the code for thesame value of R. In general, the value of R and W must be carefullychosen to get the best benefit of the coding operation. As mentionedearlier, another reason for the discrepancy could be the use of TCPReno.

FIG. 16 plots the variation of goodput with the size of the codingwindow size W. The loss rate for this plot is 5%, with the redundancyfactor fixed at 1.06. We see that the best coding window size is 2. Notethat a coding window size of W=1 corresponds to a repetition code thatsimply transmits every packet 1.06 times on average. In comparison, asimple sliding window code with W=2 brings a big gain in throughput bymaking the added redundancy more useful. However, going beyond 2 reducesthe goodput because a large value of W can mislead TCP into believingthat the capacity is larger than it really is, which leads to timeouts.The best value of W for this setup is usually 2 for a loss rate up toaround 5%, and is 3 for higher loss rates up to 25%. Besides the lossrate, the value of W could also depend on other factors such as theround-trip time of the path.

FIG. 17 shows the goodput as a function of the packet loss rate. Foreach loss rate, the values of R and W have been chosen by trial anderror, to be the one that maximizes the goodput. We see that in thelossless case, TCP performs better than TCP/NC. This could be because ofthe computational overhead that is introduced by the coding and decodingoperations, and also the coding header overhead. However, as the lossrate increases, the benefits of coding begin to outweigh the overhead.The goodput of TCP/NC is therefore higher than TCP. Coding allows lossesto be masked from TCP, and hence the fall in goodput is more gradualwith coding than without. The performance can be improved further byimproving the efficiency of the computation.

The device(s) or computer systems that integrate with the processor(s)may include, for example, a personal computer(s), workstation(s) (e.g.,Sun, HP), personal digital assistant(s) (PDA(s)), handheld device(s)such as cellular telephone(s), laptop(s), handheld computer(s), oranother device(s) capable of being integrated with a processor(s) thatmay operate as provided herein. Accordingly, the devices provided hereinare not exhaustive and are provided for illustration and not limitation.

References to “a microprocessor” and “a processor”, or “themicroprocessor” and “the processor,” may be understood to include one ormore microprocessors that may communicate in a stand-alone and/or adistributed environment(s), and may thus be configured to communicatevia wired or wireless communications with other processors, where suchone or more processor may be configured to operate on one or moreprocessor-controlled devices that may be similar or different devices.Use of such “microprocessor” or “processor” terminology may thus also beunderstood to include a central processing unit, an arithmetic logicunit, an application-specific integrated circuit (IC), and/or a taskengine, with such examples provided for illustration and not limitation.

Furthermore, references to memory, unless otherwise specified, mayinclude one or more processor-readable and accessible memory elementsand/or components that may be internal to the processor-controlleddevice, external to the processor-controlled device, and/or may beaccessed via a wired or wireless network using a variety ofcommunications protocols, and unless otherwise specified, may bearranged to include a combination of external and internal memorydevices, where such memory may be contiguous and/or partitioned based onthe application. Accordingly, references to a database may be understoodto include one or more memory associations, where such references mayinclude commercially available database products (e.g., SQL, Informix,Oracle) and also proprietary databases, and may also include otherstructures for associating memory such as links, queues, graphs, trees,with such structures provided for illustration and not limitation.

References to a network, unless provided otherwise, may include one ormore intranets and/or the internet, as well as a virtual network.References herein to microprocessor instructions ormicroprocessor-executable instructions, in accordance with the above,may be understood to include programmable hardware.

Unless otherwise stated, use of the word “substantially” may beconstrued to include a precise relationship, condition, arrangement,orientation, and/or other characteristic, and deviations thereof asunderstood by one of ordinary skill in the art, to the extent that suchdeviations do not materially affect the disclosed methods and systems.

Throughout the entirety of the present disclosure, use of the articles“a” or “an” to modify a noun may be understood to be used forconvenience and to include one, or more than one of the modified noun,unless otherwise specifically stated.

Elements, components, modules, and/or parts thereof that are describedand/or otherwise portrayed through the figures to communicate with, beassociated with, and/or be based on, something else, may be understoodto so communicate, be associated with, and or be based on in a directand/or indirect manner, unless otherwise stipulated herein.

Although the methods and systems have been described relative to aspecific embodiment thereof, they are not so limited. Obviously manymodifications and variations may become apparent in light of the aboveteachings. Many additional changes in the details, materials, andarrangement of parts, herein described and illustrated, may be made bythose skilled in the art.

Having described preferred embodiments of the invention it will nowbecome apparent to those of ordinary skill in the art that otherembodiments incorporating these concepts may be used. Additionally, thesoftware included as part of the invention may be embodied in a computerprogram product that includes a computer useable medium. For example,such a computer usable medium can include a readable memory device, suchas a hard drive device, a CD-ROM, a DVD-ROM, or a computer diskette,having computer readable program code segments stored thereon. Thecomputer readable medium can also include a communications link, eitheroptical, wired, or wireless, having program code segments carriedthereon as digital or analog signals. Accordingly, it is submitted thatthat the invention should not be limited to the described embodimentsbut rather should be limited only by the spirit and scope of theappended claims.

What is claimed is:
 1. A computer-implemented method in which atransmitting node performs operations comprising: determining a linearcombination of packets to transmit from a transmit queue; transmittingsaid linear combination of packets across a network using a slidingwindow protocol; and receiving an acknowledgement (ACK), wherein apacket is acknowledged when a receiving node receives the linearcombination of packets and determines which packet of said linearcombination of packets has been newly seen, wherein said determiningwhether a packet is newly seen includes retrieving coding coefficientsfrom a header of said linear combination of packets, appending saidcoding coefficients to a basis matrix, and performing a Gaussianelimination to determine which packet is newly seen so that the newlyseen packet can be acknowledged.
 2. The method of claim 1 furthercomprising removing a packet from said transmit queue which has beenacknowledged.
 3. The method of claim 1 wherein said network comprises anetwork operating in accordance with a Transmission Control Protocol(TCP).
 4. The method of claim 1 wherein said transmitting said linearcombination of packets across a network using a sliding window protocolfurther comprises including coding coefficients in a header of saidlinear combination of packets.
 5. The method of claim 1 wherein saidlinear combination of packets is linearly independent of previouslytransmitted linear combinations of packets and therefore conveys newinformation.
 6. The method of claim 1 wherein said ACK comprisesreceiving a request of a byte sequence number of a first byte of a firstunseen packet.
 7. A computer-implemented method in which a receivingnode performs operations comprising: receiving a linear combination ofpackets across a network using a sliding window protocol; determiningwhether a packet is newly seen, wherein said determining whether apacket is newly seen includes retrieving coding coefficients from aheader of said linear combination of packets, appending said codingcoefficients to a basis matrix, and performing a Gaussian elimination todetermine which packet is newly seen so that the newly seen packet canbe acknowledged; and transmitting an acknowledgement (ACK), wherein apacket is acknowledged when the receiving node receives the linearcombination of packets and determines which packet of said linearcombination of packets has been newly seen.
 8. The method of claim 7wherein said network comprises a network operating in accordance with aTransmission Control Protocol (TCP).
 9. The method of claim 7 furthercomprising maintaining a buffer of linear combinations of packets thathave not been decoded yet.
 10. The method of claim 7 further comprisingdelivering decoded packets to a node.
 11. The method of claim 7 whereinsaid ACK comprises receiving a request of a byte sequence number of afirst byte of a first unseen packet.
 12. A computer-implemented methodin which an intermediate node performs operations comprising: receivinga linear combination of packets; determining a linear combination ofpackets to transmit from a transmit queue; transmitting said linearcombination of packets across a network using a sliding window protocol;and when said intermediate node receives an innovative packet,determining from said innovative packet a newly seen packet and addingthe witness of the newly seen packet to said transmit queue, whereinsaid determining whether a packet is newly seen includes retrievingcoding coefficients from a header of said linear combination of packets,appending said coding coefficients to a basis matrix, and performing aGaussian elimination to determine which packet is newly seen so that thenewly seen packet can be acknowledged.
 13. The method of claim 12further comprising determining if a packet is no longer involved inreceived linear combinations of packets, and removing said packet fromsaid transmit queue.
 14. The method of claim 12 wherein said networkcomprises a network operating in accordance with a Transmission ControlProtocol (TCP).
 15. A non-transitory computer readable storage mediumhaving computer readable code thereon for providing network coding basedflow control, the medium including instructions in which a transmittingnode performs operations comprising: determining a linear combination ofpackets to transmit from a transmit queue; transmitting said linearcombination of packets across a network using a sliding window protocol;and receiving an acknowledgement (ACK), wherein a packet is acknowledgedwhen a receiving node receives the linear combination of packets anddetermines which packet of said linear combination of packets has beennewly seen, wherein said determining whether a packet is newly seenincludes retrieving coding coefficients from a header of said linearcombination of packets, appending said coding coefficients to a basismatrix, and performing a Gaussian elimination to determine which packetis newly seen so that the newly seen packet can be acknowledged.
 16. Thecomputer readable storage medium of claim 15 further comprising removinga packet from said transmit queue which has been acknowledged.
 17. Thecomputer readable storage medium of claim 15 wherein said networkcomprises a network operating in accordance with a Transmission ControlProtocol (TCP).
 18. The computer readable storage medium of claim 15wherein said transmitting said linear combination of packets across anetwork using a sliding window protocol further comprises includingcoding coefficients in a header of said linear combination of packets.19. The computer readable storage medium of claim 14 wherein said linearcombination of packets is linearly independent of previously transmittedlinear combinations of packets and therefore conveys new information.20. The computer readable storage medium of claim 14 wherein said ACKcomprises receiving a request of a byte sequence number of a first byteof a first unseen packet.
 21. A non-transitory computer readable storagemedium having computer readable code thereon for providing networkcoding based flow control, the medium including instructions in which areceiving node performs operations comprising: receiving a linearcombination of packets across a network using a sliding window protocol;determining whether a packet is newly seen, wherein said determiningwhether a packet is newly seen includes retrieving coding coefficientsfrom a header of said linear combination of packets, appending saidcoding coefficients to a basis matrix, and performing a Gaussianelimination to determine which packet is newly seen so that the newlyseen packet can be acknowledged; and transmitting an acknowledgement(ACK), wherein a packet is acknowledged when the receiving node receivesthe linear combination of packets and determines which packet of saidlinear combination of packets has been newly seen.
 22. The computerreadable storage medium of claim 21 wherein said network comprises anetwork operating in accordance with a Transmission Control Protocol(TCP).
 23. The computer readable storage medium of claim 21 furthercomprising maintaining a buffer of linear combinations of packets thathave not been decoded yet.
 24. The computer readable storage medium ofclaim 21 further comprising delivering decoded packets to a node. 25.The computer readable storage medium of claim 21 wherein said ACKcomprises receiving a request of a byte sequence number of a first byteof a first unseen packet.
 26. A non-transitory computer readable storagemedium having computer readable code thereon for providing networkcoding based flow control, the medium including instructions in which anintermediate node performs operations comprising: receiving a linearcombination of packets; determining a linear combination of packets totransmit from a transmit queue; transmitting said linear combination ofpackets across a network using a sliding window protocol; and when saidintermediate node receives an innovative packet, determining from saidinnovative packet a newly seen packet and adding the witness of thenewly seen packet to said transmit queue, wherein said determiningwhether a packet is newly seen includes retrieving coding coefficientsfrom a header of said linear combination of packets, appending saidcoding coefficients to a basis matrix, and performing a Gaussianelimination to determine which packet is newly seen so that the newlyseen packet can be acknowledged.
 27. The method of claim 26 furthercomprising determining if a packet is no longer involved in receivedlinear combinations of packets, and removing said packet from saidtransmit queue.
 28. The computer readable storage medium of claim 26wherein said network comprises a network operating in accordance with aTransmission Control Protocol (TCP).
 29. A transmitting node comprising:a memory; a processor; a communications interface; an interconnectionmechanism coupling the memory, the processor and the communicationsinterface; and wherein the memory is encoded with an applicationproviding network coding based flow control, that when performed on theprocessor, provides a process for processing information, the processcausing the transmitting node to perform the operations of: determininga linear combination of packets to transmit from a transmit queue;transmitting said linear combination of packets across a network using asliding window protocol; and receiving an acknowledgement (ACK), whereina packet is acknowledged when a receiving node receives the linearcombination of packets and determines which packet of said linearcombination of packets has been newly seen, wherein said determiningwhether a packet is newly seen includes retrieving coding coefficientsfrom a header of said linear combination of packets, appending saidcoding coefficients to a basis matrix, and performing a Gaussianelimination to determine which packet is newly seen so that the newlyseen packet can be acknowledged.
 30. The transmitting node of claim 29further comprising removing a packet from said transmit queue which hasbeen acknowledged.
 31. The transmitting node of claim 29 wherein saidnetwork comprises a network operating in accordance with a TransmissionControl Protocol (TCP).
 32. The transmitting node of claim 29 whereinsaid transmitting said linear combination of packets across a networkusing a sliding window protocol further comprises including codingcoefficients in a header of said linear combination of packets.
 33. Thetransmitting node of claim 29 wherein said linear combination of packetsis linearly independent of previously transmitted linear combinations ofpackets and therefore conveys new information.
 34. The transmitting nodeof claim 29 wherein said ACK comprises receiving a request of a bytesequence number of a first byte of a first unseen packet.
 35. Areceiving node comprising: a memory; a processor; a communicationsinterface; an interconnection mechanism coupling the memory, theprocessor and the communications interface; and wherein the memory isencoded with an application providing network coding based flow control,that when performed on the processor, provides a process for processinginformation, the process causing the receiving node to perform theoperations of: receiving a linear combination of packets across anetwork using a sliding window protocol; determining whether a packet isnewly seen, wherein said determining whether a packet is newly seenincludes retrieving coding coefficients from a header of said linearcombination of packets, appending said coding coefficients to a basismatrix, and performing a Gaussian elimination to determine which packetis newly seen so that the newly seen packet can be acknowledged; andtransmitting an acknowledgement (ACK), wherein a packet is acknowledgedwhen the receiving node receives the linear combination of packets anddetermines which packet of said linear combination of packets has beennewly seen.
 36. The receiving node of claim 35 wherein said networkcomprises a network operating in accordance with a Transmission ControlProtocol (TCP).
 37. The receiving node of claim 35 further comprisingmaintaining a buffer of linear combinations of packets that have notbeen decoded yet.
 38. The receiving node of claim 35 further comprisingdelivering decoded packets to a node.
 39. The receiving node of claim 35wherein said ACK comprises receiving a request of a byte sequence numberof a first byte of a first unseen packet.
 40. An intermediate nodecomprising: a memory; a processor; a communications interface; aninterconnection mechanism coupling the memory, the processor and thecommunications interface; and wherein the memory is encoded with anapplication providing network coding based flow control, that whenperformed on the processor, provides a process for processinginformation, the process causing the intermediate node to perform theoperations of: receiving a linear combination of packets; determining alinear combination of packets to transmit from a transmit queue;transmitting said linear combination of packets across a network using asliding window protocol; and wherein when said intermediate nodereceives an innovative packet, determining from said innovative packet anewly seen packet and adding the witness of the newly seen packet tosaid transmit queue, wherein said determining whether a packet is newlyseen includes retrieving coding coefficients from a header of saidlinear combination of packets, appending said coding coefficients to abasis matrix, and performing a Gaussian elimination to determine whichpacket is newly seen so that the newly seen packet can be acknowledged.41. The intermediate node of claim 40 further comprising determining ifa packet is no longer involved in received linear combinations ofpackets, and removing said packet from said transmit queue.
 42. Theintermediate node of claim 40 wherein said network comprises a networkoperating in accordance with a Transmission Control Protocol (TCP).